The Promise and Challenges of Integrating Real-World Evidence in Clinical Research

The role of real-world evidence (RWE) in regulatory, drug development, and healthcare decision-making is rapidly expanding. While RWE cannot substitute the evidence obtained from randomized controlled studies (RCTs), the two can be viewed as complementary sources with the same goal of understanding and improving patient's outcomes. However, the hopes of RWE have been tempered by several critical aspects/ challenges such as quality of data sources, potential for systematic bias, or formulating a research question using causal inference framework. In this session, we will discuss commonly encountered issues and recommend key methodological considerations and potential solutions for (1) assessing representativeness and generalizing results from experimental to non-experimental studies, (2) identifying under-represented groups in clinical trials for pharmacotherapy for opioid use disorder, (3) characterizing and increasing diversity in clinical trials, and (4) assessing biases and constructing valid ''synthetic control'' arms for (oncology) clinical trials. Each speaker will have 15-20 min each, followed by a 10-min discussion. Additional Q&A time will be allocated at the end of the session. The individual s are described in more detail below. (1) Ben Ackerman;Title: Using real-world data to assess representativeness and improve generalizations of study findings Randomized trials are considered the gold standard for estimating causal effects. Trial findings are often used to inform policy and programming efforts, yet their results may not generalize well to a relevant target population if the trial sample is not representative of the population of interest. More specifically, generalizations will be hindered if a trial is not similar to the population with respect to characteristics that moderate the treatment effect. Statistical methods have been developed to assess representativeness and improve generalizability by combining trials with data from non-experimental studies. Real-world datasets derived from electronic health records are promising resources that can supplement trial data when applying such methods. However, identifying the right real-world data source with the appropriate characteristics captured can be challenging in practice. In this talk, we will articulate a framework for combining trial and real-world data to assess representativeness and ultimately addressing concerns of generalizability. Through this framework, we will provide guidance on defining the target population of interest, identifying a suitable real-world data source describing that population, harmonizing across the data sources, and drawing meaningful comparisons between the trial and target population. This work will provide researchers with methods and tools to contextualize trial findings within the target population of interest through the use of real-world data. (2) Kara Rudolph;Title: Characterizing subgroups that are under-represented in clinical trials for pharmacotherapy for opioid use disorder The opioid epidemic in the United States is a public health emergency, exacerbated by the Covid-19 pandemic. Medications for opioid use disorder (MOUD)- injection naltrexone, buprenorphine, and methadone- are the most effective tools for improving outcomes and preventing overdose among persons with opioid use disorder (OUD), but engagement in MOUD, especially long-term engagement typically required for a successful outcome, is unacceptably low. Long-term engagement rates tend to be even lower in real-world settings-what National Institute on Drug Abuse (NIDA) has termed the research-to-practice gap. This discrepancy between trial and real-world MOUD effectiveness could be partially attributable to differences between clinical trial and real-world population characteristics (e.g. in terms of psychiatric and substance use comorbidities, previous treatment experience, and immigration status) if treatment effects are modified (increased/decreased) by some of these characteristics that also relate to trial participation. We identif and characterize clinically meaningful, interpretable subgroups of persons seeking OUD treatment in US usualcare settings who are not represented or underrepresented in MOUD trials based on multiple characteristics simultaneously. This moves us beyond existing approaches for assessing representation that have generally been limited to considering one individual-level characteristic at a time (e.g. race/ethnicity). (3) Madison Stoms;Title: Minority representation in clinical trials: generalizing trial results to diverse populations Since its origin, medical research has persistently lacked minority representation. In 2020 alone, the US Food and Drug Administration (FDA) reported that a mere 6% and 11% of clinical trial participants report Black and Hispanic race, respectively. Along with efforts to directly increase representation, via revised recruitment strategies, methods are being developed to leverage external data containing information on under-represented populations. The field from which these methods arise, real-world evidence (RWE), is rapidly emerging and aims to address clinical questions outside the scope of clinical trials. In this talk, we focus on generalizability and transportability methods, which can be used to extrapolate results from mostly racially homogeneous samples to diverse populations. We will also discuss the current state of diversity in clinical trials, important assumptions of generalizability and transportability methods, and applications relevant to increasing racial/ethnic representation. (4) Elizabeth Garrett-Mayer;Title: Leveraging RWD for new indications for FDA-approved anticancer agents: are we there yet? Vast amounts of health outcome data are available in real-world datasets (RWDs), like electronic health record databases and medical claim databases. In rare disease settings, performing randomized trials may be resource-intensive and inefficient due to accrual challenges. Efforts have been made to derive ''control arms'' from RWDs, representing a standard of care treatment arm, so all (or most) patients prospectively enrolled in a trial can be assigned to an experimental arm. Major challenges exist in ensuring that comparisons between the arms are meaningful, valid, and unbiased. This talk will discuss challenges, including potential for biases, harmonizing outcome measures, and efforts moving forward that will facilitate supplementing clinical trial data with RWD.

Medical bridging therapy before pulmonary endarterectomy?

Introduction: Pulmonary endarterectomy (PEA) is the recommended treatment for patients with operable chronic thromboembolic pulmonary hypertension (CTEPH). Reducing PVR pre-surgery may lower the surgical risk, but efficacy of drug treatment in operable CTEPH has not yet been proven and surgeons are concerned that dissection may be more difficult in pre-treated patients. Methodology: A randomised, double-blind, placebo controlled, multinational prospective study was performed in patients with operable CTEPH and PVR >800 dynes.sec.cm-5 at baseline (NCT03273257). Patients were randomised to Riociguat or placebo for 3 months prior to PEA. Primary endpoint was the change in PVR from baseline to before PEA. Secondary endpoints included perioperative findings and evaluation of the PEA specimen. Planned recruitment was 88 patients over 2 years. Result(s): The study was terminated early because of slow recruitment and the COVID-19 pandemic. At the time of study cessation, 14 patients were randomised (7 in each group) and 11 patients completed PEA surgery. At diagnosis, PVR was 944.0 dynes.sec.cm-5 in the Riociguat group and 1007.5 dynes.sec.cm-5 in the control group. -5 -5 The mean change in PVR prior to PEA was -28.4% for Riociguat and -6.9% for placebo (p=0.14). Completeness of surgical clearance was as expected in all patients. In the Riociguat group ease of dissection plane was rated as easier in 1, normal in 3 and more difficult in 2. In the control group, it was rated as easier in 1 and normal in 4. There were no surgical complications or post-operative deaths and no new safety signals. Conclusion(s): Due to the premature study discontinuation and the limited sample size, we are unable to determine the impact of bridging therapy on PEA outcomes.

Breakthrough COVID-19 cases and hospitalization risk: ASCO COVID-19 registry

Background: The ASCO Registry was created to analyze the impact of COVID-19 (COVID) on treatment (Tx) and outcomes of patients (pts) with cancer. Vaccination questions were added in 2021, enabling examination of breakthrough cases i.e., SARS-CoV-2 infection after vaccination. Methods: The Registry includes pts who test positive for SARS-CoV-2 and have clinically evident cancer receiving Tx or resected cancer on adjuvant Tx <12 months since surgery at the time of the positive test. Practices report data on COVID vaccine type and dates, positive SARS-CoV-2 test date, and subsequent Tx. Breakthrough cases were defined as 1) > 35 days after 1st Pfizer dose, 2) > 42 days after 1st Moderna dose, 3) > 14 days after 1st Janssen dose, or 4) > 42 days after 1st unknown vaccine dose. Data submission cutoff for all practices was February 2, 2022. Summary statistics and exact binomial 95% confidence intervals (CIs) were calculated. Results: This analysis reports on 231 pts with breakthrough cases. Median age is 68 years and 57% of pts are over age 65. Of 73 pts with non-metastatic solid tumors, 19% were hospitalized. Of 74 with metastatic solid tumors, 39% were hospitalized. Of 75 with B-cell malignancies, 32% were hospitalized. Of 9 with other hematologic malignancies, 56% were hospitalized. While the fraction of patients in the ASCO registry with breakthrough cases who were hospitalized remained fairly constant throughout 2021 (approximately 40%), those with breakthrough cases occurring in the last month of 2021 and early 2022 had a lower hospitalization rate (approximately 20%), which is consistent with less severe cases of COVID-19 in patients infected with the omicron variant (the predominant variant in the US in Dec 2021-Jan 2022) (Table). The majority of breakthrough cases occurred more than 6 months after initial vaccination (71%) and the median time to breakthrough infection from initial vaccination was 7.3 months (IQR: 5.6 9.9). The Registry continues to accrue cases and additional vaccine data;analysis will be updated with most current vaccination data for possible Annual Meeting presentation. Conclusions: Hospitalizations among patients with cancer and breakthrough COVID in the ASCO Registry occur at a high rate, particularly among patients with advanced cancers and hematologic malignancies. This decrease in recent months is likely due to increased incidence of the Omicron variant that resulted in milder COVID symptoms among those infected. A majority of SARS-CoV-2 infections occurring six months or more after vaccination suggests waning vaccine efficacy over time that could be impacted by additional doses.

Presentation, management, and outcome of COVID-19 among patients with cancer in the Middle East and North Africa (MENA) region

Background: Patients with cancer are vulnerable population that suffered during the COVID-19 pandemic from SARS-CoV-2 infection and from the pandemic's impact on healthcare systems. We are presenting the findings of MENA Registry for COVID-19 and Cancer (MRCC) regarding the SARS-CoV-2 infection presentation, diagnosis, treatment, complications, and outcomes. Methods: MRCC was adapted from ASCO COVID-19 Registry and included patients with SARS-CoV-2 infection and underlying cancer diagnosis including a newly diagnosed cancer in the work-up phase or patients with active cancer receiving cancer therapy or supportive care, or within first year of adjuvant chemotherapy or after one year of curative therapy and receiving hormonal therapy. Registry included data on patients from 12 centers in eight countries in the MENA region, namely: Saudi Arabia, Jordan, Lebanon, Turkey, Egypt, Algeria, United Arab Emirates, and Morocco. The data included patient and disease characteristics, COVID-19 presentation, management, and outcomes. The follow up is differential as data get captured at different points of disease trajectory for each patient which may not reflect the final outcome. Results: Data on 1345 patients were captured in the study by December 7, 2021. Median age was 57.1 years (18-98), whereas 56.1% were females. The median follow-up was 98.5 days (0-554). The most common COVID-19 symptoms was fever (50.3%) and 26.8% of patients were asymptomatic. Out of the 959 patients with complete data on hospitalization, 554 (57.8%) were hospitalized and 126 of them (22.7%) were admitted to intensive care unit (ICU). The majority of hospitalized patients (60%) had respiratory complications and 13.9% had sepsis and 8.5% suffered acute renal injury. As shown in Table, more than quarter of the patients died with 47% of death from COVID-19 or related complication and 60.6% died at home. More than half of the patients were fully recovered from infection. Conclusions: Although more than half of the patients recovered form COVID-19 and more are expected to recover with a longer follow up, the death toll and complications remain high in this patient population. Future analysis of the impact of vaccination and better disease management as well as the impact of newer variants would provide a useful insight on managing this vulnerable population.

COVID-19 vaccine uptake in patients with cancer and Sars-Cov-2 infection in 2020: Results from the ASCO registry

Background: The availability of safe and effective COVID-19 vaccines has enabled protections against serious outcomes from SARS-CoV-2 infection. The ASCO Registry provides an opportunity to study the rate of vaccine uptake among patients (pts) with cancer who had SARS-CoV-2 infection prior to availability of vaccines. Methods: Participating US medical oncology practices identify pts eligible for inclusion: pts with a positive SARS-CoV-2 test and, at the same time, either (a) or (b) cancer-free less than 12 months (i.e., in complete remission or post-resection) and receiving adjuvant treatment. Registry data were extracted on 2/4/22. Pts in this analysis had a SARS-CoV-2 positive test between 3/1/2020 and 12/31/2020 and at least one clinical encounter on or after 1/1/21, documenting receipt or not of 1st and 2nd vaccine shots. Pts lacking data on vaccination status in their medical records were excluded from analysis. When the exact date of 1st vaccine shot was known, the primary endpoint was number of days from 1/1/21 to 1st shot date. If the exact date of 1st shot was unknown, date was categorized as within 7 days from the clinical encounter, 2-4 weeks, or more than 1 month prior. Summary statistics and 95% confidence intervals (CIs) were calculated using Kaplan-Meier methods. Results: The Registry included 1260 pts with SARS-CoV-2 infection in 2020. Time to vaccination (described using cumulative incidence) across age categories is shown in Table, with 1/1/21 as the baseline. Median time to vaccination ranged from 63 days in the oldest pts (> 80 years of age) to 220 days in the youngest pts (< 45 years of age). Differences in rates of vaccination uptake were most pronounced in the early months (Jan - Mar) and diminished in April and beyond. Conclusions: Older pts with cancer who were infected with SARS-CoV-2 in 2020 received COVID-19 vaccinations in 2021 at higher rates than those of younger ages. Delayed uptake by younger aged pts at outset could be related to lack of clarity on pts with cancer qualifying for early vaccine access and state policies on availability.

Relationship Between Brain Functioning, Behavior, and Microbiota in Autism Spectrum Disorder

Several studies have linked these GI issues to altered gut microbial composition, or dysbiosis, in ASD. Yet research has not yet clarified how dysbiosis may be related to the core features of ASD or to the symptom heterogeneity of this disorder. To our knowledge, no studies have investigated gut-microbiome-brain-behavior interactions in a single population of individuals with ASD to better understand the mechanisms of ASD heterogeneity. By examining these interactions, we aim to test the general hypothesis that alterations in gut microbial composition are correlated with structural and functional brain alterations, as well as with clinical and behavioral features of ASD. Building on the brain imaging and behavioral data generated by the Co-PIs R01 grant, we will acquire stool samples from all participants and characterize the composition, metagenome, and metabolome of their gut microbiome. We will relate individual differences across three continuums of symptoms (GI issues, social deficits, and sensory deficits) to activity in selected brain networks and microbiome-related data. The long-term goal of this research is to explain heterogeneity, develop biomarkers, and ultimately build individualized treatments for ASD directed at brain-gut-microbiome pathways.

Different clinical trial analyses for different stakeholders

The predominant reporting of clinical trial results today is what we might call the ''regulator's analysis.'' This analysis meets requirements that regulators have set for their work. Shouldn't there also be an analysis for other stakeholders, for example, physician's analysis, a thirdparty payor's analysis, a patient's analysis, and other analyses? Different analyses for different stakeholders appearing in different journals would allow Bayesian analyses, simulation results, effectiveness data, realworld evidence, more sophisticated safety analyses, and so on to also be published. The need for analyses for different stakeholders has been heightened by the various stakeholders involved in the analysis of COVID-19 trials and the current attention to estimands and concern about the proportional hazards assumption in survival analysis. This session will present the case that ''persuasive evidence'' is defined differently by different clinical trial stakeholders and provide some examples of innovative clinical trials analyses. Chair and Org: Greg Ball, Merck [greg.ball@merck .com] Title: Is There Only One Analysis? Jay Herson, Johns Hopkins Bloomberg School of Public Health [jay.herson@ earthlink.net]. Summary: The main output of clinical trials is to yield persuasive evidence. However, our notion of persuasive evidence has changed over time. The intent-to-treat frequentist analysis preferred by regulatory agencies may no longer be considered persuasive to other stakeholders. This talk will motivate the need for and provide examples of additional analyses that are useful for other stakeholders such as physicians, patients, and insurers. Title: Safety Analysis: A Physician's Perspective Barbara Hendrickson, AbbVie [barbara.hendrickson@ abbvie.com] Summary: Clinical trial safety data presentations are typically confined to rates of different categories of adverse events (e.g. common, fatal, serious, and those leading to discontinuation). However, other types of questions are important to healthcare providers and patients. Specifically, information about time to onset, reversibility with and without intervention, and additional measures of severity by functional impairment or treatments required is frequently asked questions. Also, more meaningful risk factor analyses as well as information about the likelihood for key adverse reactions in different patient subpopulations (e.g. by age and renal impairment) are needed. Title: Efficacy Analysis: Beyond the Tyranny of the p-value Michael Gaffney, Pfizer [michael.gaffney@pfizer. com]. Summary: Clinical trials are usually large, expensive and not repeated. The adherence to statistical rules often impedes the interpretation and dissemination of the strength of clinical trial results. These rules serve the legitimate objectives of regulators and medical journal editors. However, the emergence of personalized and targeted therapy;the relatively new concept of estimands;the increasing use of adaptive designs;and real-world evidence all provide an opportunity for a new paradigm which should lead to a fuller and integrated understanding of clinical trial results. This new paradigm requires reducing the role of hypothesis testing/ decision-making and increasing the role of estimation and uncertainty. Discussant: Elizabeth Garrett-Mayer, American Society of Clinical Oncology [Liz.Garrett-Mayer@asco.org].

Changes implemented by U.S. oncologypractices in response to COVID-19 pandemic: Initialreport from the ASCO Registry on COVID-19 and cancer

Background: In April 2020, ASCO initiated a registry to capture and analyze status and outcomes of patients withcancer and COVID-19, and to describe effects of the pandemic on U.S. cancer practices. Initial findings of changesto care delivery are included. Methods: Practices provide data on changes to care delivery due to COVID-19 and longitudinal data on patients with cancer and confirmed COVID-19. At present, 26 cancer practices have enrolled in the Registry-5 academic,15 hospital/health-system (H/HS) owned, and 6 physician-owned (P-O) located in 19 states. Enrollment of practicesand data collection is ongoing. Results: Twenty sites, from 17 practices (3 academic, 9 H/HS owned, and 5 P-O in 15 states) responded (April 20-June 4). All incorporated telemedicine visits;90% reported use of telemedicine was new. 30% reported “decliningsome but not all” new patient requests. For patients with cancer not on active therapy, 15% of sites postponed someroutine visits, 35% conducted virtually all routine visits by telemedicine, and 50% used telemedicine for some routinevisits. Most sites (95%) reported following clinical guidelines for visit postponement;90% reported following localhealth authorities on when to resume routine visits. 90% screened patients prior to in-office visits for COVID-19symptoms by phone and at clinic entrance;10% screened patients using only one method. 30% modifiedintravenous (IV) drug infusions, including halting some or all (10%), shortening some or all (20%), or switching fromIV to oral drugs (15%). While no sites conducted home-based, anticancer drug infusions, 30% are considering thisoption if COVID19 conditions change. Most sites modified laboratory specimen collection, including allowing acollection site closer to home (60%) and collection in a patient's home (1 site). Two sites only allowed patients onoral anticancer drugs to use alternate collection sites. Only 1 site reported specimen collection in patients' homes.All reported making the following changes to clinic arrangements: requiring use of masks, eliminatingaccompaniment by a support person (with exceptions), and reducing the visit numbers or increasing time betweenvisits. No sites reported shortages of anticancer or supportive care drugs. 45% experienced shortages ofnasopharyngeal swabs, 45% of medical hand sanitizer, and 75% of personal protective equipment. 40% of siteshave experienced staffing reductions or changes due to reduced patient visits (30%), transfer to other clinical areas(20%), availability (15%), and COVID-19 illness (15%). Conclusions: The COVID-19 pandemic has had a substantial impact on most aspects of cancer care delivery inU.S. oncology practices. All practices incorporated telemedicine, which is new to most. Adjustments were made topatient visits and scheduled IV drug infusions. Sites reported shortages of equipment related to COVID-19, notcancer or supportive care drug shortages. At the time of the AACR meeting we expect to have data from morepractices.

Impact of COVID-19 pandemic on newpatient and consult visits at 20 hematology/oncologypractices in the ASCO PracticeNET learning network

Introduction: During March 2020, as the emergence of COVID-19 began to influence medical and social behaviorsin the United States, oncology practices reported a disruption in normal referral and patient management patterns.Participants and staff of ASCO's PracticeNET learning network sought to explore and quantify the impact of thisdisruption through an analysis of patient activity at multiple hematology/oncology practices. Methods: 20 practices submitted their billing data for analysis;practices were located in 14 states and ranged insize from 2 to 29 hematologists/oncologists. From this dataset we analyzed a total of 11,453 new patient and consultvisits (Current Procedural Terminology codes 99201-99205, 99241-99245, 99251-99255, and 99341-99345)performed by hematologists/oncologists from February 9 to April 18, 2020. The number of visits performed fromFebruary 9 to March 14, 2020 was compared to visits performed from March 15 to April 18, 2020. A principaldiagnosis was assigned to each visit following usual coding and billing practices. Results: From February 9 to March 14, practices performed an average of 70.1 (median 55.5) new patient andconsult visits per week. From March 15 to April 19, practices performed an average of 44.5 new patient and consultvisits per week. The average decrease in visits among practices was 35% (95% confidence interval (CI): -42%,-29%). The decline in visits per practice ranged from -61% to -13%. New patient and consult visits for solidneoplasms decreased by an average of 22% (95% CI: -31%, -13%), visits for blood neoplasms decreased by anaverage of 36% (95% CI: -25%, -47%), and visits for benign hematology and circulatory disorders decreased by anaverage of 44% (95% CI: -53%, -34%). Conclusions: Oncology practices experienced a decline in new patient and consult visits, first observed in the weekof March 15. New patient and consult visits for blood neoplasms, benign hematology, and circulatory disordersexperienced a greater decline than visits for solid neoplasms. The decrease in activity could be the result ofprioritization of resources or changes in patient behavior in seeking care. Further study is necessary to quantify theimpact of these findings on patient access and outcomes and to monitor recovery efforts.

Differences in level-of-service fortelehealth visits as compared to in-office visits for cancerand hematology patients seen in practices within theASCO PracticeNET learning network

Introduction: In response to the need for social distancing and infection prevention during the COVID-19 pandemic, there has been increased use of telehealth services to manage cancer and hematology patients. Throughout Marchand April of 2020, the Medicare and Medicaid programs expanded coverage of telehealth services, allowing cancerand hematology patients to receive certain telehealth services from their home during the public health emergency.We analyzed data from ASCO's PracticeNET learning network to examine the reported level-of-service fortelehealth services compared to standard in-office visits. Methods: 20 practices submitted their billing data for analysis;practices were located in 14 states and ranged insize from 2 to 29 hematologists/oncologists. We analyzed a total of 33,435 established patient evaluation andmanagement visits performed by hematologists/oncologists from March 15 to April 18, 2020. 3,062 (9.1%) visits were performed via telehealth and 30,373 were performed in a physician office or outpatient hospital department.The level-of-service of each visit was identified through the reported Current Procedure Terminology (CPT) code, where levels 1-5 correspond to CPT codes 99211-99215, respectively, and level 5 represents the highest complexityvisit. Telehealth visits were identified through use of the modifiers 95, GQ, and GT, as appended to the applicableCPT code. Results: The level-of-service distribution for telehealth-based visits was level 1 (1%), level 2 (4%), level 3 (35%), level 4 (50%), and level 5 (11%). This contrasted with in-office visits: level 1 (3%), level 2 (2%), level 3 (27%), level 4(51%), and level 5 (18%). Differences were greatest in level 3 visits (35% vs. 27%) and level 5 visits (11% vs. 18%).Differences in level-of-service persisted when exploring various disease cohorts, including patients with solidneoplasms, blood neoplasms, benign hematology disorders, and circulatory disorders. Conclusions: Analysis of established patient visits showed that telehealth visits were reported at lower level-of-service as compared to in-office visits. This finding may be related to directing straightforward visits to be performedvia telehealth or due to limitations in using telehealth by patients with complex medical problems.